Aggregation Methods for Lineary-solvable Markov Decision Process ⋆
نویسندگان
چکیده
A general class of stochastic optimal control problems has recently been reduced to computing the principle eigenfunction of a linear operator. Here we present an approximation framework for solving such problems by using soft state aggregation over a continuous space. This approach enables us to avoid matrix factorization and take advantage of sparsity by using efficient iterative solvers. Adaptive schemes for basis placement are developed so as to provide higher resolution at the regions of state space that are visited more often. Numerical results on test problems are provided.
منابع مشابه
State aggregation for solving Markov decision problems an application to mobile robotics
In this paper, we present two state aggregation methods, used to build stochastic plans, modelling our environment with Markov Decision Processes. Classical methods used to compute stochastic plans are highly untractable for problems necessiting a large number of states, like our robotics application. The use of aggregation techniques allows to reduce the number of states to take into account, ...
متن کاملHierarchical Linearly-Solvable Markov Decision Problems
We present a hierarchical reinforcement learning framework that formulates each task in the hierarchy as a special type of Markov decision process for which the Bellman equation is linear and has analytical solution. Problems of this type, called linearly-solvable MDPs (LMDPs) have interesting properties that can be exploited in a hierarchical setting, such as efficient learning of the optimal ...
متن کاملHierarchy through Composition with Linearly Solvable Markov Decision Processes
Hierarchical architectures are critical to the scalability of reinforcement learning methods. Current hierarchical frameworks execute actions serially, with macroactions comprising sequences of primitive actions. We propose a novel alternative to these control hierarchies based on concurrent execution of many actions in parallel. Our scheme uses the concurrent compositionality provided by the l...
متن کاملAggregation and disaggregation in Markov decision models for inventory control
AGGREGATION AND DISAGGREGATION IN MARKOV DECISION MODELS FOR INVENTORY CONTROL
متن کاملOptimizing Red Blood Cells Consumption Using Markov Decision Process
In healthcare systems, one of the important actions is related to perishable products such as red blood cells (RBCs) units that its consumption management in different periods can contribute greatly to the optimality of the system. In this paper, main goal is to enhance the ability of medical community to organize the RBCs units’ consumption in way to deliver the unit order timely with a focus ...
متن کامل